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The Intercollegiate Studies Institute report, The Shaping of the American Mind: The 
Diverging Influences of the College Degree & Civic Learning on American Beliefs, sug- 
gests that college is failing to provide an adequate education in civic knowledge and is al- 
so influencing graduates to become less supportive of American values. Desirable “civic 
learning” about American values is associated with positions and attitudes that are anti- 
abortion, pro-free-market economics, consistent with fundamentalist Biblical interpreta- 
tions, and otherwise generally associated with contemporary conservative political posi- 
tions. The bulk of the report focuses on contrasts of college graduates and non-graduates 
on these political values, based on a civics test administered as a telephone survey. “Civ- 
ics knowledge,” as they define it, is presented as of greater value than college education. 
The study suggests that more educated people are more liberal, yet the omission of basic 
information about the researchers’ polling methods, their fundamental analytic techniques 
and their actual results renders any such conclusions insupportable. Consequently, the re- 
port offers no valid information that could inform policy makers or the public. 



Summary of Review 




Review 



I. Introduction 

On February 14, 2010, Fox News aired a 
segment on a new report by the Intercolle- 
giate Studies Institute (ISI), The Shaping of 
the American Mind: The Diverging Influ- 
ences of the College Degree & Civic Learn- 
ing on American Beliefs .* ISI is a think ta nk 
dedicated to furthering college students’ 
“understanding of the economic, political, 
and moral principles that sustain a free and 
humane society” including limited govern- 
ment, the rule of law, market economy, and 
moral norms (p. 30). The report paints a pic- 
ture of college graduates as ill-informed and 
out of step with American values, criticizing 
their poor civic knowledge and liberal lean- 
ings on social issues. 2 The image presented 
prompted host Clayton Morris to ask Tucker 
Carlson on Fox and Friends, “Is it better to 
not go to college at all than to get this kind 
of education?” 3 

II. Findings and Conclusions 
of the Report 

The report identifies four major findings. 
The first states that college fails to adequate- 
ly transmit civic knowledge. Seventy-one 
percent of Americans “failed” the ISI’s civic 
knowledge test, with college graduates aver- 
aging 57% correct and non-college gra- 
duates scoring 49% correct. 4 

The first finding also identifies five social 
issues where college “influenced” opinions. 
Those with college degrees were more likely 
to favor same-sex marriage and abortion on 
demand. They were less likely to believe 
that anyone can succeed in America with 
hard work, to favor teacher-led prayer in 
public schools, and to believe that the Bible 
is the “Word of God.” The report found it 



“surprising” that college graduates were not 
more supportive of the relevance of Ameri- 
ca’s Founding documents. 

The second major finding examined the “in- 
fluence” of civic knowledge and a college 
degree on social issues opinions. Civic 
knowledge “affected” personal opinions on 
20 of the issues (of which the authors hig- 
hlighted six), compared to the above- 
mentioned five “affected” by earning a ba- 
chelor’s degree. Those scoring higher on the 
civic knowledge test were more likely to 
agree that (a) a person’s evaluation of a na- 
tion improves with his or her understanding 
of its structures; (b) prosperity depends on 
entrepreneurs and the free market; and (c) 
the Ten Commandments remain relevant. 
They were less likely to agree (a) that legis- 
latures should subsidize colleges in propor- 
tion to their students learning about Ameri- 
ca; (b) that the free market brings about full 
employment; and (c) that the Bible is the 
Word of God. Less support by those with 
more civic knowledge for prayer in school 
was among the 13 items not addressed. 

The third major finding was that civic know- 
ledge, not college, increased a person’s re- 
gard for America’s ideals. The authors iden- 
tified four issues out of 39 as opposing 
America’s ideals: (a) believing that America 
corrupts otherwise good people, (b) believ- 
ing that America’s Founding documents are 
obsolete, (c) believing that the Ten Com- 
mandments are irrelevant, and (d) disagree- 
ing with the statement that prosperity de- 
pends on entrepreneurs and free market. For 
these four items, having a higher score on 
the civic knowledge test had a “significant 
impact,” but a college degree did not. 

According to the report, younger people in- 
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dicated less support for these American 
ideals. However, the report suggested that 
“the solution to this problem is not simply 
sending more young Americans to college, 
for college itself makes little difference in a 
person’s views on these matters” (p. 19). 

Finally, of the 2,508 national public survey 
respondents, 240 had taught a college-level 
course, and they were more likely to agree 
that America corrupts otherwise good 
people, that the Ten Commandments are ir- 
relevant, that raising the minimum wage de- 
creases employment, that educators should 
instill more doubt in students and reject cer- 
tainty, and that homeschooling families neg- 
lect their community obligations. For none 
of these statements did college instructors 
agree and non-college instructors disagree; 
professors just had higher levels of agree- 
ment. 

III. The Report’s Use of 
Research Literature 

Although there is a massive literature on 
civic education and the learning of political 
values, the report’s scant literature review 
focuses on Benjamin Franklin’s philosophy 
regarding the purpose of higher education 
and brief reviews of two previous studies 
conducted by ISI itself. The underlying 
premise of the study is that a major purpose 
of higher education is to teach history, gov- 
ernment, capitalism and democracy. The 
authors interpret Franklin’s (and George 
Washington’s) vision of higher education as 
a program designed to “nurture a national 
consensus built around those enduring prin- 
ciples necessary to maintaining a free, pros- 
perous, and self-governing nation.” Howev- 
er, the report does not recognize or address 
the reality that others disagree even with the 
ISI interpretation of Franklin’s ideas. For 
instance, professors Harkavy and Hartley at 
the University of Pennsylvania, which was 



founded by Franklin, consider Franklin’s 
vision as providing direction for collabora- 
tions with communities to provide demo- 
cratic opportunities to the less fortunate. 5 
And, of course, Fra nk lin was not the only 
wise or knowledgeable person to weigh in 
on issues such as civic education, the pur- 
pose of higher education, or the learning of 
political values. 

IV. Review of the Report’s 
Methods 

Instruments 

Of the 33 items on the ISI civic knowledge 
test, three were from the test required for 
U.S. citizenship and three approximated 
items from that test. 6 The remaining items 
were considerably more difficult than the 
citizenship test. For instance, nine of the 33 
items were under the category “market 
economy” and dealt with issues like pro- 
gressive tax structures, international trade, 
and fiscal policy. 7 Moreover, the items about 
abortion legislation, the Scopes Monkey 
Trail, Aquinas, and the Federal Reserve may 
not fit commonly accepted notions of basic 
“civic knowledge.” 

Sample 

In a telephone interview, 2,508 respondents 
answered a 3 3 -item mostly multiple-choice 
test and a 39-item, five-part Likert-type 
scale opinion survey. Respondents also pro- 
vided 46 pieces of demographic information. 
Considering the entire telephone survey was 
1 1 8 questions, only the most dedicated res- 
pondents would make it to the end, calling 
into serious question the unsubstantiated 
claim in the report that these participants 
“represent a probability sample of all indi- 
viduals who live in households with residen- 
tial telephone service in the United States” 
(p. 24). The introductory script was not pro- 
vided, nor was the response rate to the sur- 
vey or the actual results. That is, although 



http://epicpolicv.org/thinktank/review-shaping-American-mind . 



3 of 7 



readers are told that 2,508 respondents ans- 
wered the survey, readers are not told 
whether 3,000 were initially called or if that 
number was 5,000, or even 20,000. The 
lower the response rate, the greater chance 
of selection bias and therefore invalid re- 
sults. 

The authors did mention that a standard 
weighting process was used as part of the 
sampling process. However, neither the de- 
mographic makeup of the sample nor the 
weights were presented. For example, there 
was no mention of the number of college 
and non-college graduates in the sample — a 
vital factor for a survey contrasting the val- 
ues of these two groups. Furthermore, the 
nature of the sample comes into question 
when readers are told that almost 10% of the 
respondents were college instructors. Of the 
228 million adults in the U.S. in 2008, 1.7 
million were instructors at the post- 

o 

secondary level (less than 1%). 

Analyses 

The report included a page of statistical equ- 
ations to demonstrate the multiple regression 
approach used. Although the page includes 
technical terminology, no actual results from 
the use of these equations were presented. 
As discussed below, the most vital omission 
was effect sizes; no effect sizes were pre- 
sented. The reader, therefore, can have no 
notion of whether the purported relation- 
ships were large or trivial. 

V. Review of the Validity of 

the Findings and Conclusions 

Due to the lack of actual results reported in 
any meaningful way, the reader is left with a 
collection of unsupported assertions. 

Colleges should inculcate 
a particular “civics education” 

The implicit premise of this study is that a 



primary purpose of a college education is to 
indoctrinate students to a particular interpre- 
tation of democracy. Based on the report, it 
appears that this desired form of democracy 
must emphasize founding documents, capi- 
talism and free market economics, and 
Christianity (particularly the Old Testa- 
ment). This bias is pervasive and appears to 
be linked to the stated purpose of the 1ST 

Improper polling and test scoring 

ISI’s survey, which was lengthy at 118 
items, may or may not have generated credi- 
ble results. Because of the omitted informa- 
tion about response rates and completion 
rates, the reader cannot know. Moreover, on 
the subset of 33 civic knowledge questions, 
setting the passing point at 60% on a test of 
very specific knowledge, where multiple- 
choice items with up to five response choic- 
es were read over the phone, appears arbi- 
trary. No rationale for this cut-off score was 
provided. The first major finding was that 
college did not improve civic knowledge; 
however, college graduates outscored non- 
college graduates by eight percentage points. 
Without external validation benchmarks, 
standard deviations, sample sizes, and the 
conventionally accepted statistical tests, it is 
impossible to know whether that difference 
was statistically significant or particularly 
meaningful. 

Causal inferences 

The authors repeatedly and improperly in- 
ferred causation from multiple regressions (a 
correlation-based approach). Having a col- 
lege degree may be related to scores on a 
test or responses to opinion statements, but 
that does not mean that the college degree 
caused these outcomes. Further, failing to 
report the size and significance of this corre- 
lation leaves the reader adrift. People who 
choose to attend college may share different 
characteristics than those who do not attend. 
People who attend conservative Bob Jones 
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University may share different characteris- 
tics than those who attend UC Berkeley, and 
although the experiences of the students at 
those schools may certainly affect their civic 
knowledge and their political attitudes, a 
great deal of those differences and characte- 
ristics undoubtedly exist at the time they en- 
ter as freshmen. The data and analyses used 
in this report cannot support causal state- 
ments regarding the influence of college, yet 
that was the crux of the report. 

Selective and contradictory findings 

Respondents with college degrees reported a 
lower level of agreement with the statement 
that the Bible is the Word of God. But so did 
those with higher civic knowledge. Escaping 
the narrative were the results that those with 
higher civic knowledge scores expressed 
less support for prayer in school, less sup- 
port for the free market’s ability to produce 
full employment, and less agreement that 
American Founders would oppose universal 
healthcare. This sort of selective emphasis 
of findings again points to the report’s bias. 

Missing statistics 

The biggest flaw in the report was what was 
not there: statistics. Considering the rather 
technical description of the methodology, 
the lack of statistics was surprising and casts 
pervasive doubt on the analysis. 

As mentioned, the results from the multiple 
regressions were not reported, nor were the 
significance of the equations or the weights 
for the variables. Also missing were even 
basic numbers such as sub-sample sizes 
(e.g., college degree and not), means, and 
standard deviations. The approach used to 
interpret answers is also troubling and ap- 
pears to have resulted in exaggerated differ- 
ences. Using the 1 to 5 Likert scale (with 3 
as neutral), anything less than 2.99 was ca- 
tegorized as “disagree” and 3.01 and above 
was “agree.” Yet, the actual means were not 



How ISI Defines Basic Civics 

Ten things you should know to pass 
the ISI civic knowledge test: 

1. What aspect of slavery was the 
main topic of the Lincoln- 
Douglas debates? 

2. What does the Bill of Rights pro- 
hibit concerning discrimination 
based on sex and the official reli- 
gion of the United States? 

3 . What did Roosevelt threaten when 
the Supreme Court found parts of 
the New Deal unconstitutional? 

4. What was the impact of the Anti- 
Federalists on the US Constitution? 

5 . What was the impact of Plessy v. 
Ferguson, Roe v. Wade, and the 
National Organization of Women 
on abortion? 

6. What philosophy do Socrates, Pla- 
to, Aristotle, and Aquinas share? 

7. What did the Puritans believe 

concerning the morality of war, 

religious freedom, and the sinful- 
ness of humanity? 

8. What is the definition of business 
profit? 

9. What does level of localization of 

knowledge in the price system 

have to do with economic prosperi- 
ty based on free markets versus the 
government's centralized planning? 

10. What policy tool, other than con- 
trolling interest rates, does the 
Federal Reserve have? 

Source: Summarized from the ISI ISI civic 
knowledge test for The Shaping of the 
American Mind: The Diverging Influ- 
ences of the College Degree & Civic 
Learning on American Beliefs , pp. 26, 27. 



reported. In the report, the items were listed 
in order of strength of opinion, but another 
table listed the items by the size of their 
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standard deviation (without reporting the 
standard deviation). Other than an occasion- 
al percentage score, the report was devoid of 
any supporting detail. 

VI. Usefulness of the Report 
for Guidance of Policy 
and Practice 

The reporting of this study could easily lead a 
casual reader to erroneously infer that higher 
education diminishes an individual’s civic 
knowledge. The study’s design and presenta- 
tion also suggest that contemporary conserva- 
tive political values are the equivalent of civ- 
ic knowledge. The result is an ill-supported, 
anti-intellectual conclusion, as demonstrated 
by Fox News’ Tucker Carlson’s answer 
about whether a student should go to college: 
“Well that’s actually an open question.” 

The resulting policy guidance this report 
provides may, in fact, be destructive of the 
very ideals of education the authors ascribe 
to the Founding Fathers — particularly in- 
formed democratic participation. Based on 
decades of sound research, earning a college 
degree has been associated with improved 



general verbal and quantitative skills, specif- 
ic subject matter knowledge, oral and writ- 
ten communication skills, and formal rea- 
soning and critical thinking. 9 This is in addi- 
tion to the well-established benefits of im- 
proved occupational opportunities and high- 
er income. 10 

Methodologically, the report is overly nar- 
row in ascribing a purpose to higher educa- 
tion, the civic knowledge test is not repre- 
sentative of the field, the 118-question tele- 
phone interview instrument is suspect, and 
the actual results were not presented. 

The report seeks to promote a particular po- 
litical position in the guise of social science. 
It therefore fails Albert Einstein’s definition 
of civic learning in a democracy: 

What can the schools do to defend 
democracy? Should they preach a 
specific political doctrine? I believe 
they should not. If they are able to 
teach young people to have a critical 
mind and a socially oriented attitude, 
they will have done all that is neces- 
sary. 1 1 
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